Multiple Linear Regression¶


  • In the previous chapter we learned about predicting the output (Y) for single input (X).
  • In this chapter, we are going to learn about predicting the output (Y) for multiple inputs ($X_n$)

Example : Predicting the price of house (Y) on the basis of multiple input : $X_1$ Nummber of bedrooms , $X_2$ age of property , $X_3$ size of property .

Comparision between Single and multiple Linear Regression¶

No description has been provided for this image
  • In 1 input problem $m$ and $c$ are evaluated to represent a line in the 2D space.

  • In 2 input problem $m_1$ ,$m_2$ and $c$ are evaluated to represent a 2D plane in the 3D space.

For 3 or more input features.¶

-For 3 input problem we need to evaluate ($m_1$ ,$m_2$ ,$m_3$ and $c$ )to represent 3D hyper- plane in a 4dimensional space .
-Similary for more than 3 input features.
-Although it is not visualizable we can evaluate best fit $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ according to the number of input features $X_n$

ChatGPT Image Apr 21, 2025, 12_16_59 AM

The process of evalulating best fit $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ is same as that of Simple Linear Regression.
$i.e$ Minimizing the Sum of Squared Error.

For one variable $m_1$
$Step 1$ : Evaluation of partial derivative of SSE with respect to $m_1$
Partial Derivative of SSE with respect to $m_1$ is given by :
$P$= $\frac{\partial }{\partial m_1 }SSE$

$P$= $\frac{\partial }{\partial m_1 }\sum(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c))^2$
( Here Summation deontes this opeartion to be performed for each data point and all outputs are added )
$P$= $\sum\frac{\partial }{\partial m_1}(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c))^2 $
$P$= $ \sum2(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c) )(-x_1) $

$Step 2$: Make small step
$new$ $m_1$ = $m_1$ + $k$* $P$ where k is constant called step size. As the name suggests it determine the size of each step .

$Step 3$: Step 1 and 2 are performed for each variables $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ .

$Step 4$: Step 1 ,2,3 repetedly until $P$ attends value almost near to zero for each variables $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ .

Out[2]:
No description has been provided for this image
No description has been provided for this image

Implementation in Real-World Problem¶

Imports¶

In [3]:
import numpy as np
import pandas as pd
import matplotlib as mp
from matplotlib import pyplot as plt
from sklearn.linear_model import LinearRegression

Data¶

This data from a book An Introduction to Statistical Learning
The dataset contain 3 input features :
Amout of $s ( in thousand of dollor ) spent in advertising through

  1. Television
  2. Radio
  3. Newspaper.

The output is the total unit of sales ( thousand of unit) .

In [4]:
data_path = "https://www.statlearning.com/s/Advertising.csv" 

# Read the CSV data from the link
data= pd.read_csv(data_path,index_col=0)

# Print out first 5 samples from the DataFrame
data.head()
Out[4]:
TV radio newspaper sales
1 230.1 37.8 69.2 22.1
2 44.5 39.3 45.1 10.4
3 17.2 45.9 69.3 9.3
4 151.5 41.3 58.5 18.5
5 180.8 10.8 58.4 12.9

Implementing multiple Linear Regression in this data.¶

In [5]:
# Prepare da
X = data[['TV', 'radio', 'newspaper']]
y = data['sales']

# Fit multiple linear regression model
model = LinearRegression()
model.fit(X, y)

# Print coefficients and intercept
print(f'm1 (TV coefficient): {model.coef_[0]:.3f}')
print(f'm2 (Radio coefficient): {model.coef_[1]:.3f}')
print(f'm3 (Newspaper coefficient): {model.coef_[2]:.3f}')
print(f'c (Intercept): {model.intercept_:.2f}')
m1 (TV coefficient): 0.046
m2 (Radio coefficient): 0.189
m3 (Newspaper coefficient): -0.001
c (Intercept): 2.94

  • Here value of m1, m2, m3 means the numerical representaiton of importance of that specific input feature to predict the output .

Predicting for a new data¶

In [6]:
new_data = pd.DataFrame({
    'TV': [50],
    'radio': [60],
    'newspaper': [70]
}) 

predicted_sales = model.predict(new_data)
print('The predicted number of sales is :', predicted_sales, ' thousand')
The predicted number of sales is : [16.46629814]  thousand
In [ ]: